Systems and methods for human-machine subconscious data exploration

ABSTRACT

Systems, computer readable media, and method concern includes generating data renderings for a data set. The data renderings for the data set include one or more of visual renderings of portions of the data set and one or more sonic renderings of portions of the data set. The method further includes providing the data renderings to a user via one or more output devices. The method also includes capturing biofeedback data from the user using one or more human interface devices. The biofeedback data includes biological responses to the one or more data renderings. Further, the method includes continuously generating and providing new data renderings based on the biofeedback data. The new data renderings incorporate features in the data renderings identified from the biofeedback data. The method also includes determining one or more features of interest in the data set based on the biofeedback data and the new data renderings.

CLAIM OF PRIORITY

This application claims priority to U.S. Provisional Patent Application62/253,459 filed on Nov. 10, 2015, which is hereby incorporated byreference in its entirety

BACKGROUND

As computers have evolved, they have become ever more powerful whileshrinking in size. Meanwhile, a global communication/data system knownas the Internet has been constructed that allows users anywhere in theworld to connect and share data. Today, globally-connectedsemiautonomous data machines can be found in everything from mobilephones to thermostats. This explosion of constantly connected, powerful,ubiquitous, miniature computers has allowed for the creation of enormousdata generating and mining networks. Companies can now collect data onalmost every aspect of their operations from delivery routes to powerusage or clients' webclicks. These companies can use these “Big Data”sets to optimize operations and increase value. In fact, there hasarisen an entire discipline devoted to this called BusinessIntelligence.

Even though individuals and companies have access to huge amounts ofdata, all of this data may not be useful for all analysis. The sheersize and power of the infrastructure necessary to manage and analyze BigData sets requires unique solutions. In addition, the size of the BigData sets requires creative problem solving in order to begin to be ableto analyze the data. Currently, Business Intelligence solutions revolvearound trying to make the data more manageable for analysis. Inparticular, analysts are developing algorithms that only use the“important parts” of the Big Data sets or are designing models that canuse multiple computers for analysis.

As exponentially more samples from the physical and cyber worlds arecollected and fed into a data stream, the chances of keeping up with thedata rate are slimming rapidly. Today, data analysts almost exclusivelyrely on machine learning techniques. These applications transform theraw data into lower dimensional feature spaces that capture differentaspects of the input stream. While often existing methods cannot scaleto the current large volumes of data, recent theoretical advancements ofrandomized algorithms may enable approximate variants to be directlyapplicable to large-scale data. This, however, may not be enough: theincremental improvement in machine learning is only a small step towardthe goal of rapid knowledge extraction.

The number of methods available for knowledge extraction has reached apoint where making the correct choice of method by itself may become amajor issue. Some methods perform better on certain types of complexproblems, while others are more robust to omnipresent artifacts. Thepurpose of these different algorithms is to aid the learning experienceand to speed up discoveries, but the aggregate output of all thesemethodologies is again too much for any analyst to fully comprehend.Computers can perform any one of an infinite number of possibleprojections or non-linear embeddings, but unsupervised programs do notknow where to look in the data. Moreover, the computers cannotdifferentiate obvious patterns from emerging discoveries. Furthermore,analysts must constantly assess the validity, correctness, and relevanceof analytical results, while they attempt to fully understand thesituation reflected in the data. As such, the efficiency of Big Dataprocessing algorithms, and the systems implementing them, need to beimproved by improving the quality of analytical feedback from ananalyst, and providing the improved feedback to the algorithms faster.

SUMMARY

Aspects of the present disclosure concern a method that includesgenerating data renderings for a data set. The data renderings for thedata set include one or more of visual renderings of portions of thedata set and one or more sonic renderings of portions of the data set.The method further includes providing the data renderings to a user viaone or more output devices. The method also includes capturingbiofeedback data from the user using one or more human interfacedevices. The biofeedback data includes biological responses to the oneor more data renderings. Further, the method includes continuouslygenerating and providing new data renderings based on the biofeedbackdata. The new data renderings incorporate features in the datarenderings identified from the biofeedback data. The method alsoincludes determining one or more features of interest in the data setbased on the biofeedback data and the new data renderings.

Additional aspects of the present disclosure concern a system thatincludes one or more output devices, one or more human interfacedevices, and a computer system. The computer system includes one or morememory devices storing instructions, and one or more processors coupledto the one or more memory devices and configured to execute theinstructions to perform a method. The method includes generating datarenderings for a data set. The data renderings for the data set includeone or more of visual renderings of portions of the data set and one ormore sonic renderings of portions of the data set. The method furtherincludes providing the data renderings to a user via one or more outputdevices. The method also includes capturing biofeedback data from theuser using one or more human interface devices. The biofeedback dataincludes biological responses to the one or more data renderings.Further, the method includes continuously generating and providing newdata renderings based on the biofeedback data. The new data renderingsincorporate features in the data renderings identified from thebiofeedback data. The method also includes determining one or morefeatures of interest in the data set based on the biofeedback data andthe new data renderings.

Additional aspects of the present disclosure concern a non-transitorycomputer readable medium storing instructions for causing one or moreprocessors to perform a method. The method includes generating datarenderings for a data set. The data renderings for the data set includeone or more of visual renderings of portions of the data set and one ormore sonic renderings of portions of the data set. The method furtherincludes providing the data renderings to a user via one or more outputdevices. The method also includes capturing biofeedback data from theuser using one or more human interface devices. The biofeedback dataincludes biological responses to the one or more data renderings.Further, the method includes continuously generating and providing newdata renderings based on the biofeedback data. The new data renderingsincorporate features in the data renderings identified from thebiofeedback data. The method also includes determining one or morefeatures of interest in the data set based on the biofeedback data andthe new data renderings.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 illustrates an example of a human-machine co-learning system,according to various aspects of the present disclosure.

FIG. 2 illustrate an example of a processes of human-machineco-learning, according to various aspects of the present disclosure.

FIGS. 3A and 3B illustrate one example of human-machine co-learningsystem and processes, according to various aspects of the presentdisclosure.

FIGS. 4A-4C illustrate an example of data and results for human-machineco-learning processes of FIGS. 3A and 3B, according to various aspectsof the present disclosure.

FIG. 5 illustrates an example of a hardware configuration for a computerdevice, according to various aspects of the present disclosure.

DETAILED DESCRIPTION

For simplicity and illustrative purposes, the principles of the presentteachings are described by referring mainly to examples of variousimplementations thereof. However, one of ordinary skill in the art wouldreadily recognize that the same principles are equally applicable to,and can be implemented in, all types of information and systems, andthat any such variations do not depart from the true spirit and scope ofthe present teachings. Moreover, in the following detailed description,references are made to the accompanying figures, which illustratespecific examples of various implementations. Logical and structuralchanges can be made to the examples of the various implementationswithout departing from the spirit and scope of the present teachings.The following detailed description is, therefore, not to be taken in alimiting sense and the scope of the present teachings is defined by theappended claims and their equivalents.

Aspects of the present disclosure are directed to a human-machineco-learning system that allows the cooperative analysis of data by auser and a computer system. Human intuition may not work in higher thanthree dimensions, and the human-machine co-learning system can extractknowledge from the available data. The human-machine co-learning systemcloses the loop between computer systems and users by collectingfeedback representing the user's reactions to the data renderings. Thehuman-machine co-learning system alters how users (e.g., data analysts)interact with data by leveraging the most evolved aspects of humanintuition and pairing them with the thoroughness of moderncomputing/machine learning. In aspects, the human-machine co-learningsystem provides a constant multi-channel synchronous feedback loop thatconnects the computer system to the user. The human-machine co-learningsystem creates varying data renderings (e.g., visualizations,sonifications, physical stimulus, etc.), and presents the datarenderings to a user. As the data renderings are presented to the user,the human-machine co-learning system simultaneously collects the user'sreactions (biofeedback) to the data renderings (e.g., brain waves,eye-movements, micro facial expressions, etc.). In real-time, thehuman-machine co-learning system incorporates biofeedback into one ormore statistical models behind the data rendering. This human-machineco-learning system enables an effective exploration in the parameterspace and an effortless navigation in the data.

FIG. 1 illustrates a human-machine co-learning system 100 in which datacan be analyzed, according to aspects of the present disclosure. WhileFIG. 1 illustrates various components contained in the human-machineco-learning system 100, FIG. 1 illustrates one example of ahuman-machine co-learning system and additional components can be addedand existing components can be removed.

As illustrated in FIG. 1, the human-machine co-learning system 100includes a computer system 102. The human-machine co-learning system 100can represent the computer systems and network hardware of public orprivate entities, such as governmental agencies, individuals,businesses, partnerships, companies, corporations, etc., utilized tosupport the entities. The computer system 102 can be any type ofconventional computer systems that is operating in the human-machineco-learning system 100 or supporting the human-machine co-learningsystem 100. For example, the computer system 102 can include varioustypes of servers, such as file servers, web servers, applicationservers, database servers, email servers and the like, that provideservices within the human-machine co-learning system 100. Likewise, forexample, the computer system 102 can include laptop computers, desktopcomputers, tablet computers, mobile phones, and the like used by thepersonnel of the entities.

Additionally, for example, the human-machine co-learning system 100includes other hardware and computer systems that support thehuman-machine co-learning system 100. For example, the human-machineco-learning system 100 can include gateways, routers, wireless accesspoints, firewalls, and the like that support any type of communicationsnetworks to allow the computing systems in the human-machine co-learningsystem 100 to communicate. In any of the examples, the computer systems,including the computer system 102, in the human-machine co-learningsystem 100 include hardware resources, such as processors, memory,network hardware, storage devices, and the like, and software resources,such as operating systems (OS), application programs, and the like.

The computer system 102 can be coupled to a repository 104. Therepository 104 can be configured to store data that is associated withand utilized by the human-machine co-learning system 100. The repository104 can be implemented using any type of type of storage media. Therepository 104 can store the data in any type of format that is utilizedby the human-machine co-learning system 100.

The computer system 102 can also be coupled to one or more repositories106 via one or more networks 108. The computer system 102 can beconfigured to retrieve or receive, via a network 108, the data used bythe human-machine co-learning system 100 from the repositories 106. Thenetwork 108 can be any type of network whether public or private. Therepository 106 can be any type of computer system that storesinformation about the components.

According to aspects of the present disclosure, the computer system 102is configured to execute a data analysis tool 110. The data analysistool 110 is configured to create varying data renderings 112 (e.g.,visualizations, sonifications, physical stimulus, etc.), and present thedata renderings to a user 114. The data analysis tool 110 is configuredto provide the data renderings to the user 114 using one or more dataoutput devices 116 coupled to the computer system 102. The data analysistool 110 is configured to perform data rendering and analysis techniqueswith biofeedback data 118 collected from the user 114 using one or morehuman interface devices 120. The data analysis tool 110 is configured toprovide a framework for cooperative learning where statistical learningalgorithms provide the user 114 with tailored data renderings 112 basedon the current interest of the user 114 using a combination of logic andsubconscious intuition determined from the biofeedback data 118. Byusing the biofeedback data 118 from the human interface devices 120, thehuman-machine co-learning system 100 provides improved data analysis byspeeding up the interaction between the user 114 and the computer system102 and enabling advanced data exploration in higher dimensions thanpossible with standard data review.

In aspects, the data analysis tool 110 is configured as a softwareprogram that is capable of being stored on and executed by the computersystem 102. The data analysis tool 110 can be written in a variety ofprogramming languages, such as JAVA, C++, Python code, Visual Basic,hypertext markup language (HTML), extensible markup language (XML), andthe like to accommodate a variety of operating systems, computing systemarchitectures, etc.

The data analysis tool 110 is configured to operate, at least partially,under the control of a user 114. The data analysis tool 110 can beinitiated and configured by the user 114. For example, the user 114 canselect data to be analyzed by the data analysis tool 110, can selecttypes of data analysis to be performed by the data analysis tool 110,can select and configure the data output devices 116 and human interfacedevices 120, and the like. The data analysis tool 110 can output resultsof the data analysis to the user 114 or any other system or user in thehuman-machine co-learning system 100. To communicate with the user 114,the data analysis tool 110 is configured to generate and provide one ormore user interfaces. The user interfaces can be any type of commandline and/or graphical user interface (GUI) that allows the user 114 tointeract with the data analysis tool 110. The data analysis tool 110 isconfigured to provide, via the user interfaces, controls, forms,reports, etc., to allow the user 114 to interact with the data analysistool 110 and perform the processes described herein.

In aspects, the human-machine co-learning system 100, controlled by oneor more users 114, can process one or more data sets. For example, thedata analysis tool 110 can evaluate data in the data set to identifyparameters and potential embeddings, and produces one or more datarenderings 112, (e.g., visual representations, audio representations,physical stimulus etc.) of the data. The data analysis tool 110 canprovide the data renderings to the data output devices 116 that areconsumed by the user 114.

In aspects, the data output devices 116 can include any type of outputdevice to provide the data renderings 112 to the user 114. For example,the data output devices 116 can include one or more computer displays ormonitors for displaying the visual renderings of the data renderings112. The one or more computer displays or monitors can be any type orconfiguration, for example, liquid crystal displays (LCDs), plasmadisplays, cathode ray tube (CRT) displays, and the like.

Likewise, for example, the data output devices 116 can include one ormore audio output devices. The audio output devices can output sonicrenderings of the data renderings of the data set. The audio outputdevices can include any type and configuration of audio device.

For example, sonficiation and music-based feature selection can beexamples of data rendering that can create a more immersive andefficient environment. In its simplest form, the human-machineco-learning system 100 can use sound (e.g., music, sound effects, etc.)to drive aspects of the data rendering. For example, the data analysistool 110 can map different harmonics in the music to different featuresin the data set. With the rhythmic periodic changes of the music, thebrain of the user 114 can anticipate the new information, which makesrapid changes in the renderings easier to interpret. Exploiting thebrain's unique timing ability, the data analysis tool 110 can pinpointinteresting new features in time that can be flagged and later revisited(e.g., mapped to more prominent modes of the data rendering). Forexample, the data analysis tool 110 can map different features of thedata to different Fourier harmonics. The rendered features changerapidly with the music, yet, they can be perceived well when they areexpected due to the user 114 brain's unique timing ability. Inconnection with brain wave measurements, discussed further below, thedata analysis tool 110 can accurately identify the relevant features intime even if they are shown just briefly but periodically. Thesefeatures can be mapped to more frequent harmonics or more prominentchannels of data rendering for further analysis.

Likewise, for example, the data analysis tool 110 can actively shape thesound to channel additional features to the user 114. Active shaping ofsound can enable the data analysis tool 110 to continuously deliveradditional features to the user 114. For example, in utilizingsonification, the data analysis tool 110 can utilize a concept isfamiliarity with the soundscapes. For example, stadiums, concert hallsor living rooms reverberate the sounds differently, and these subtlechanges are accurately picked up by the human ear. By applying theseeffects to any audio recording, the data analysis tool 110 can representlower frequency, contextual information. Likewise, the data analysistool 110 can map higher frequency signals to other familiar soundeffects, such as the noises inside a moving car, driving on asphalt orgravel and the raindrops on the windshield. For example, most can easilyrelate to a situation where a person is driving a car and suddenly hearsa siren. The audio cue tells the person that something is occurring inour environment, the human experience identifies the siren as anemergency vehicle and provides information about appropriate driverresponses to emergency vehicles. If this sonification and visualizationwas applied to data rendering presented to the user 114, the user 114can begin visually scanning the environment (e.g., the visualization ofthe data renderings 112) using audio clues to localize the emergencyvehicle and determine its probable pathway to decide what actions totake as the driver of a vehicle. In the example, the user 114 seamlesslyperforms the analysis to provide an integrated, real-time assessment ofthe current environment which can be captured by the data analysis tool110. The data analysis tool 110 can apply this concept to the perceptionand understanding of big data to utilize the human ability to seamlesslyintegrate input from multiple senses to rapidly make assessment aboutwhat is occurring in the surrounding environment. As such, thehuman-machine co-learning system 100 can create a more immersive andmore efficient environment to study vast amounts of data.

Likewise, for example, the data output devices 116 can include otherdevices to output other physical stimuli to the user 114. For example,the data output devices 116 can include haptic devices (e.g.,vibration), environment control devices (e.g., heating, cooling etc.)and the like.

As the user 114 consumes the data renderings, the human interfacedevices 120 detects and records, as biofeedback data 118, the user 114various reactions to the data renderings 112, and the data analysis tool110 can incorporate the biofeedback data 118 into the data evaluationprocess. For example, immediate biofeedback data 118 from the user 114can drive the real time creation of the data renderings 112 by steeringthrough the parameter space of possible embeddings. Monitoring the user114 behavior and response can provide useful information about ongoingthought processes of the user during the data analysis.

In aspects, the human interface devices 120 can capture signalsrepresenting conscious and subconscious reactions of the user 114 to thedata renderings 112. The human interface devices 120 can be any type ofdevice that is capable of sensing or detecting actions, reactions,inputs, and the like of the user 114. For example, the human interfacedevices 120 can be configured to capture the biofeedback data 118including brain wave data (e.g., electroencephalogram (EEG) data), eyemovement tracking data, micro-facial expressions data, body language,vital sign data (heart rate, blood pressure, respiration rate, etc.),muscle movement data, capillary dilation data, skin conductivity data,and the like.

For example, the human interface devices 120 can include one or morebrain-computer interfaces (BCis) that monitor brain waves of theoperator. In some aspects, the BCis can be non-invasive to the operator,for example, using conductive electrodes placed on the scalp of the user114 to detect microvoltscale electrical potentials created by manysimultaneously active neurons in the cortex. Consumer-grade EEG devices,for example, can deliver high-resolution temporal information that canbe adequate to detect event-related evoked potentials.

Likewise, for example, the human interface devices 120 can include BCIsthat collect electro-oculogram (EOG) data, electromyogram (EMG), andcombination thereof, in addition to action potentials of the peripheralportions of the cranial nerves outside the skull, produced by suchreflexes. The action potentials can be detected through skin recordingsof bio-potentials. One example of a BCI that can detect EEG, EOG, andEMG is a neural impulse actuator. In this example, the neural impulseactuator can have the following specifications: head band withintegrated left and right sensors, silver chloride (AgCI) medical-gradeelectrodes, and center reference sensor with common mode rejection usinginversion of left and right signals; single channel recording;resolution capable of identifying 0.1 microvolt potentials; 4 kHzsampling rate; universal serial bus (USB) 2.0 interface with signaltransmission speed of at least 217 packets/sec; and host processingdevice having a multi-core processor with processing speed of at least1.5 GHz. In another example, the single channel recording can bereplaced with dual-channel recording, also adding electrode-amplifiermodules that improve signal-to-noise ratio, a wireless interface betweenheadband and host processing device, and hardware-based (e.g.,synthesized application-specific integrated circuit) signal processingto resolve left-to-right brain hemisphere asymmetries.

Likewise, for example, the human interface devices 120 can include eyemovement tracking devices. The eye movement tracking devices can utilizeone or more light sources, one or more cameras, and the like to captureand track the movement of the user 114 eyes, even as the positioning ofthe user 114 head constantly changes. For example, the eye movementtracking devices can image the user 114 eyes in a wide spectrum ofwavelengths, from the visible to infrared, to monitor the position ofthe pupils (bright in the infrared range) in addition to the reflectionsof light sources (bright in the visible range). In one example, the eyemovement tracking devices can use high-definition cameras, to resolvethe axis of each eye independently, and to use the angular disparity todetermine the z-axis position, thereby localizing the point of interesteven in the third dimension.

Likewise, for example, the human interface devices 120 can includecomputer input devices such as mice, keyboards, microphones, and thelike. For example, rapidly changing low-dimensional embeddings in thedata renderings 112 represent a path of a randomized walk in theparameter space that is primarily driven by the inputs of the user 114.The human interface devices 120 can include simple interface inputs toenhance the currently displayed scene or move out to examine the broadcontext. These advanced “grand tours” can offer more variations andangles of view than just linear global projections of the data, and thedata analysis tool 110 can create and display them without explicitrequests. For example, with a mouse-click, the user 114 can pin downapparent phenomena around a point/event to explore, and the appropriateembedding strategies can be immediately selected. Another simple controlcan affect the speed of the motion along the path, the rate of theinformation flow. In addition to traditional data exploration tools, theemphasis here is on a simple interface that drives an advanced analysisengine behind the data renderings 112. Such a design can be extended tomake use of other types of inputs that are less taxing on the user 114and faster in response. The interface can also work independently of thedimensionality of the data, which is often too large to be manuallyexplored anyway.

The human-machine co-learning system 100 can correlate the biofeedbackdata 118 with the data renderings 112. For example, based on thereceived biofeedback data 118, the data analysis tool 110 can identifyand enhance local signatures and their immediate surroundings. Using thelocal signature and surrounding, the data analysis tool 110 can uncoverhidden topological relations among the data set forming the datarenderings 112, while more global views can provide better contextualinformation. As the user 114 considers competing hypotheses, thehuman-machine co-learning system 100 can automatically present the datarenderings in a timely manner. The data analysis tool 110 canautomatically render feature vectors into a familiar environment, wherethe user 114 can subconsciously make the objective choices withouteffort and without blurring the results with false rationales that maybe based on selective or insufficient facts.

To enable a user 114 to visually navigate a data set in substantiallyreal-time, the human-machine co-learning system 100 implements acombination of techniques pertaining to machine learning, data renderingstreaming, and biofeedback acquisition. For example, the data analysistool 110 can implement a special class of randomized algorithms thatwork directly on the data renderings 112 and the biofeedback data 118using incremental strategies. As new data for the data renderings 112and/or the biofeedback data 118 arrives, the data analysis tool 110updates internal models (with or without storing the data) and goes onto fetch the next data in line. For example, the data analysis tool 110can utilize algorithms including clustering or regression analyses,low-dimensional embeddings (e.g., Principal Component Analysis (PCA)).In one example, the data analysis tool 110 can utilize incremental androbust PCA generalization. These summary statistics provide differentinsights into the raw data.

In another example, the data analysis tool 110 can utilize algorithmsand process called Rapid Serial Visual Presentation (RSVP). In RSVP, theuser 114 is presented with data renderings 112 including a series ofimages in rapid succession while the biofeedback data 118 includingneural activity is monitored. If the user 114 looks for a particulartype of visual content in the image stream, the neural activityfollowing the matching content will differ from the baseline behavior.This difference can be traced back with high precision to the triggeringimage in the data renderings 112. The different neural activity canmanifest in absence of any physical response, and can be elicited atimage presentation rates far exceeding those to which a human couldnormally respond. The data analysis tool 110 can control aspects of thedata renderings 112 using the amplitudes of signals in one or more ofthe wave bands. Interpretation of the data renderings 112 facilitatesfast and seamless navigation of the data set in several dimensions. Inaddition, the combination of high spatial resolution of the eye-trackingand the accurate timing information from the brain waves cover a widerange of human responses that can interact with advanced statisticalembeddings in an efficient synthesis, promoting an expedited cooperativelearning process. Precision timing information from brain waves alsoopens up a whole new world of possibilities for other sensory modalitiesbeyond visual and audible modalities.

FIG. 2 illustrates an example of a process 200 for analyzing data usinghuman-machine co-learning, according to aspects of the presentdisclosure. While FIG. 2 illustrates various stages that can beperformed, stages can be removed and additional stages can be added.Likewise, the order of the illustrated stages can be performed in anyorder.

After the process begins, in 202, the human-machine co-learning systemis initiated. For example, the user 114 (or other user) can initiateexecution of the data analysis tool 110 on the computer system 102.Likewise, for example, the user 114 (or other user) can initiate set upand configure the data output devices 116 and the human interfacedevices 120. In one example, the data analysis tool 110 can provide theuser interface to the user 114 (or other user). The user interface canallow the user 114 (or other user) to operate the data analysis tool110.

In 204, a data set for analysis can be determined. For example, the dataanalysis tool 110 can receive or retrieve one or more data sets from therepository 104, the repository 106, or both. Likewise, for example, thedata analysis tool 110 can receive or retrieve the one or more data setsfrom other sources in real-time. In any example, the user 114 (or otheruser) can select the one or more data sets for analysis. Likewise, thedata analysis tool 110 can automatically select one or more data setsfor analysis.

In any example, the data set can include any type of data desired to beanalyzed to locate embedding, pattern, or information of relevance orsignificance. For example, the data set can include geographic data,traffic pattern data, network usage traffic data, security information,financial data, voting data, polling data, and the like. While severalexamples of data are disclosed herein, any type of data can be analyzedwith the human-machine co-learning system 100.

In 206, the data analysis tool 110 generates data renderings for thedata set. The data analysis tool 110 can generate any type of renderingto gain biofeedback from the user 114. For example, the data analysistool 110 can generate visual data, audio data, other physical stimuli,and combinations thereof.

In 208, the data analysis tool 110 provides the data renderings to theuser 114. The data analysis tool 110 can provide the data renderings 112to the user 114 via the data output devices 116. The data analysis tool110 (and other systems and software of the computer system 102) canconvert the data renderings 112 to one or more signals that are usableby the data output devices 116.

In 210, the data analysis tool 110 captures biofeedback data from theuser 114. The data analysis tool 110 (and other systems and software ofthe computer system 102) can communicate with the human interfacedevices 120 to receive or retrieve the biofeedback data 118. Thebiofeedback data 118 can be any type of data representing the actions,reactions, physical state, etc. while consuming the data renderings 112.For example, the biofeedback data 118 can include brain wave data, eyemovement tracking data, micro-facial expressions data, body language,vital sign data, muscle movement data, capillary dilation data, skinconductivity data, and the like.

In 212, the data analysis tool 110 generates new data renderings basedon the biofeedback data. As the user 114 consumes the data renderings,the human interface devices 120 detects and records, as biofeedback data118, the user 114 various reactions to the data renderings 112, and thedata analysis tool 110 can incorporate the biofeedback data 118 into thedata evaluation process. For example, immediate biofeedback data 118from the user 114 can drive the real time creation of the datarenderings 112 by steering through the parameter space of possibleembeddings. Monitoring the user 114 behavior and response can provideuseful information about ongoing thought processes of the user duringthe data analysis.

For example, based on the received biofeedback data 118, the dataanalysis tool 110 can identify and enhance local signatures and theirimmediate surroundings. Using the local signature and surrounding, thedata analysis tool 110 can uncover hidden topological relations amongthe data set forming the data renderings 112, while more global viewscan provide better contextual information. As the user 114 considerscompeting hypotheses, the human-machine co-learning system 100 canautomatically present the data renderings in a timely manner. The dataanalysis tool 110 can automatically render feature vectors into afamiliar environment, where the user 114 can subconsciously make theobjective choices without effort and without blurring the results withfalse rationales that may be based on selective or insufficient facts.

In 214, the data analysis tool 110 provides the new data renderings tothe user 114. The data analysis tool 110 can provide the data renderings112 to the user 114 via the data output devices 116. The data analysistool 110 (and other systems and software of the computer system 102) canconvert the data renderings 112 to one or more signals that are usableby the data output devices 116.

In 216, the data analysis tool 110 determines whether to continue theanalysis. For example, the data analysis tool 110 can continue topresent data renderings 112 to the user 114 and collect the biofeedbackdata 118 until a solution is determine, the user 114 (or other user)stops the analysis, or combination of both. The data analysis tool 110provides a constant multi-channel synchronous feedback loop thatconnects the computer system 102 to the user 114. As the new datarenderings 112 are presented to the user 114, the data analysis tool 110simultaneously collects the biofeedback data 118. In real-time, the dataanalysis tool 110 incorporates biofeedback into one or more statisticalmodels behind the data rendering.

For example, the data analysis tool 110 implement eye-tracking, asdiscussed above, to precisely identify the region of interest in a largefield of view. Following the movement of the eyes of the user 114, thedata analysis tool 110 can accurately pinpoint the operator's gaze intwo- or three dimensions, and flag these interesting places. Interactivevisualization can explore these targeted areas, and gradually shifttoward “better” views until the focus of the user 114 moves to otherfeatures and trends. Such user-flagged regions of interest can be usedto help train classifiers more effectively. For instance, similar toheat maps of football fields during a game, the data analysis tool 110can create maps of most looked-at places. Over time as the embeddingsevolve, the data analysis tool 110 can automatically find the unusuallocations in the multi-dimensional feature space.

In another example, the data analysis tool 110 can utilize statisticalalgorithms continuously generate varying data renderings 112 of the dataset. The data analysis tool 110 can apply the statistical algorithmsaccording to the context that is currently being examined. Instead ofjust creating global solutions, the data analysis tool 110 enables theuser 114 to restrict the analysis to a smaller volume, if restrictionenhances the features. As such, the data analysis tool 110 allow theuser to “zoom in” and “zoom out” on the data renderings not only interms of visualization of the data but in terms of the domain of theanalysis. For example, the data analysis tool 110 can apply the conceptsof triage analytics, which focus on high level aggregates, anddrill-down analytics, available upon request for specific areas of data(e.g., “zoom in” and “zoom out”). The data analysis tool 110 can applythe “zoom in” and “zoom out” procedures to the concept of usingresources to perform detailed analysis when an operator is following atrain of thought. Thus, the user 114 can designate a focus area in anyprojection, and the data analysis tool 110 can immediately applyadvanced algorithms to incrementally enhance the data renderings 112.The data analysis tool 110 can also utilize algorithms that select andapply the optimal machine learning methodology or combination ofmethodologies.

In another example, when the user 114 is viewing the data renderings112, sensory stimuli outside of normal patterns can elicit autonomousnervous system responses, e.g., the pupil reflex, the eye lid reflex andscanning eye movements. The data analysis tool 110 can identify specificcomposite bio-potential signatures, from the biofeedback data 118detected by the human interface devices 120, based on the temporalsequence of discrete responses at the different frequency bands that areanalyzed. The data analysis tool 110 can be enabled to map independentwave bands in the alpha and beta ranges to any kind of user action. Inother words, the data analysis tool 110 can grow ostensible“brainfingers” that correspond to signals in the wave bands, and canlearn to use them to steer the data renderings toward interestingfeatures. A subconscious random walk in several dimensions can map outmore details in data than thousands of mouse clicks. This ishigh-dimensional navigation without the complication of the missinghuman intuition in high dimensions.

In another example, the data analysis tool 110 can utilize RSVPalgorithms and processes. In RSVP, the user 114 is presented with datarenderings 112 including a series of images in rapid succession whilethe biofeedback data 118 including neural activity is monitored. If theuser 114 looks for a particular type of visual content in the imagestream, the neural activity following the matching content will differfrom the baseline behavior. This difference, called “P300” due to itspositive polarity and 300-millisecond latency, can reliably detected bydigital signal processing techniques, and can be traced back with highprecision to the triggering image in the data renderings 112. For thepurposes of a BCI, because the P300 is a product of the subconsciousprocessing of visual content, it can manifest in absence of any physicalresponse, and can be elicited at image presentation rates far exceedingthose to which a human could normally respond, up to 50 Hz. Thisresponse is approximately 100 times faster than what could be achievedwithout BCI. Several independent brain wave bands can be defined inknown frequency ranges, e.g., alpha, beta. The data analysis tool 110can control aspects of the data renderings 112 using the amplitudes ofsignals in one or more of the wave bands. Interpretation of the datarenderings facilitates fast and seamless navigation of the data set inseveral dimensions. In addition, the combination of high spatialresolution of the eye-tracking and the accurate timing information fromthe brain waves cover a wide range of human responses that can interactwith advanced statistical embeddings in an efficient synthesis,promoting an expedited cooperative learning process. Precision timinginformation from brain waves also opens up a whole new world ofpossibilities for other sensory modalities beyond visual and audiblemodalities.

It is worth re-emphasizing one of the aspects of this approach: theimportance of context, the questions the operator is seeking to answer.People generally look at data with specific questions in mind. Does itmatch what I am looking for? Is it different from what I normally see?Can I see any patterns in the data? Also, people will look at the samedata but with different questions in mind, and see different things, andcome to different conclusions. For example, when someone is looking tobuy a new car, some of the data they look at will be the same regardlessof why they are buying the car. Other data, however, can be contextuallydependent upon the reason they are buying the car. If someone is buyinga used car for a 16-year old novice driver, the queries and data searchpathways will be different than if they are looking to buy a car forthemselves as a fun recreational vehicle. The context of the questionscan be important as the data that is being queried, and the results maybe determined by the context just as by the data itself. The “zoom inand zoom out of domains” approach can allows the human-machineco-learning system 100 to adapt the inquiry pathway to a specificcontext at that moment, rather than forcing the user 114 alongpredetermined inquiry pathways governed by a limited framework ofoptions. By moving away from pre-defined analytic options, “zoom in andzoom out of domain” can open up many previously unexplored avenues ofinquiry.

Returning to FIG. 2, in 218, the data analysis tool 110 determines andprovides the results of the data analysis. The results of the dataanalysis can be any meaningful features of interest, embeddings,patterns, or information that is contained within the data.

In one example, the result of the data analysis can be utilized to trainone or more machine learning algorithm. For example, as the datarenderings 112 are generated and the biofeedback data 118, the dataanalysis tool 110 can analyze the user 114 response to the datarenderings 112 and utilize the response to train one or more machinelearning algorithm. Thus, the data analysis tool 110 can improve thefuture analysis of similar sets of data.

The data analysis tool 110 can provide the results to the user 114 orother user of the human-machine co-learning system 100. The dataanalysis tool 110 can store the results, for example, in the repository104, the repository, other data stores, or combinations thereof. Thedata analysis tool 110 can provide the results to other computer system,for example, via the network 108.

FIGS. 3A, 3B, and 4A-C illustrate one example of the human-machineco-learning system 300 using the process 200 described above, accordingto various aspects. In this example, the human-machine co-learningsystem 300 implements RSVP as a fast, human-in-the-loop classificationand search tool for visualizations of large data sets. As RSVP hastraditionally been used to process photographs and images of the naturalworld, the first test of the system was a search application over asynthetic and completely digital dataset.

Referring to FIGS. 3A and 3B, the human-machine co-learning system 300implements a feedback loop around an RSVP system in order to demonstratethe power of human selection in a data set search. By using locallybiased steering around views triggered by RSVP, the user 114 can quicklysteer a search over a visual, high-dimensional dataset. Previous RSVPexperiments show the ability of the user 114 to trigger on a fixeddefinition of a target image; however, in a dataset exploration settingthe user 114 may not yet know exactly what to look for. A dynamicdefinition of a target allows the user 114 to discover what isinteresting and steer an exploratory path through the data renderings112 of the set of data. With a simple RSVP feedback loop, thehuman-machine co-learning system 300 establishes that a human subject isable to adapt his threshold of what is “interesting” as a searchprogresses.

The human-machine co-learning system 300 include a brain wave detector302, for example, a traditional wet EEG cap with 64 electrodes made byBioSemi. The brain wave detector 302 may require preparation, e.g., gelin the hair and a locked door behind which 3 people can run theanalysis. Suitable dry caps, which are more usable in an officeenvironment, can be used. An example is the Cognionics wireless72-channel dry electrode headset. This headset collects data from up to64 dry electrodes, but allows for 8 extra channels of data to beconcurrently passed along with the EEG data.

The human-machine co-learning system 300 can include one or more eyetracking devices 304. One example of an eye tracking device 304 is theEye link 1000 Plus produced by SR Research. This is a high-end,precision eye tracker which can be used to pinpoint exact areas ofinterest as well as track pupil dilation, saccades, and micro-saccades.Another example of an eye tracking device is the EyeTribe Eye Tracker, amore affordable and mobile solution that only provides low-resolutiondata on eye activity. The EyeTribe can be used to sense the level ofuser engagement. The eye tracking device 304 can provide a plethora ofdigital features which, when synchronized with the data renderings, willaid in data classification and processing.

The human-machine co-learning system 300 can include a feedback loop308, operated by the data analysis tool 110 executing on a computersystem 306, directed by EEG signals and signals collected by other typesof sensors during RSVP sessions. For example, the human interfacedevices can include sensors that are hands-free, require no physicaltethers to a computer, and are easily operated in a normal officesetting. The eye tracking device 304 can provide higher-resolution dataon area of interest of the user 114 than can be achieved with an EEG capalone. The human-machine co-learning system 300 can use the eye trackingdevice 304 to sense the level of user engagement; the RSVP dataexploration session may be paused when the user 114 is distracted orlooking away from the screen.

As shown in FIG. 3B, the feedback loop 308 connects a computer system306 to the user 114 via the human interface devices 310 that detectssignals (e.g., via the above described systems and additional sensors,such as photodiodes attached to a stimulation tracking module), collectsprocessed signal information, and presents images to the operator via adisplay 312. The data analysis tool 110 can include a data streammanagement layer 314, such as the Lab Streaming Layer (LSL) toolkit, isan appropriate framework for sensor data harvesting and synchronization.LSL can be compatible with each of the sensors described above and istherefore an example of a sensor signal harvesting platform.

Referring to FIG. 4A, a test data set included a 30 point set sampledfrom a wire-frame of a three-dimensional cube. The test data set wasthen extended to include noise in the form of additional points in the30 parameter space and in additional dimensions. For example, animplementation includes appending two random numbers after the (x,y,z)coordinates of a point, which yields a five dimensional dataset. Arandom 50 rotation is applied to these points, mixing the coordinates sothat the structure of the cube is lost in the noise: any axis-parallelviews show only noise, as shown in inset (A) of FIG. 4A. The goal of thehuman-machine co-learning system 300 can be to recover the hidden cubeby finding the correct transformation using RSVP feedback from the user114. This exercise demonstrates that the human-machine co-learningsystem 300 does not have to operate using a specified cost function orgoodness of measure to derive a solution, but can solely rely on thepower of the human brain.

Using the process 200 described above, the human-machine co-learningsystem 300 begins the analysis by performing random rotations on thesimulated dataset and rendering the rotations to for data renderings 112and provide the data renderings to the display 312 according to RSVP.The data renderings 112 can contain noise. The data renderings 112 ormovie of flicking images is just the “war of ants,” in which thehigh-dimensional dataset appears to be just noise. On rare occasions,however, the random projections will have some structure, which thebrain will detect. As soon the operator encounters a rendered plot thatcontains some structure, see inset (B) of FIG. 4A, the human inputdevices of the feedback loop (e.g., an eye tracking device 304 and abrain wave detector 302 of detect an event, such as a P300 wave or aneye focal movement. The human input devices transmit the detected signalto signal processors, such as an event sensor module that encodes thesignal into a readable digital event to be processed by the dataanalysis tool 110. The detected reaction effectively teaches thehuman-machine co-learning system 300 something about the data, and thehuman-machine co-learning system 300 can start biasing the machinelearning method behind the visualization toward the views flagged byRSVP. Over time, the sample of images gets enriched in structure andeventually the human-machine co-learning system 300 discovers the 30cube, as shown via the progression of insets (A)-(D) of FIG. 4A.

FIG. 4B illustrates the average error in the recovered rotation overtime. In particular, curve 400 represents the principal angle, inradians, between the rotation matrix used to hide the cube and therotation matrix of a given random rotation. The principal angledecreases with the training as the solution converges to the truetransformation. The graph shows that the principal angle between thetrue subspace containing the cube and the solution shrinks quickly.However, data rendering in an RSVP setting poses a number of challengesthat should not be overlooked. Plots that can be processed at high ratesby the brain are potentially quite different from traditionalvisualizations, and generating such visualizations in real-time for RSVP(10 Hz) might also prove to be computationally challenging. For example,in a first iteration of the experiment, the operator can be tasked toidentify, or trigger on, data plots showing structure while ignoringplots that only show noise. The results show that the method in whichthe data is rendered can be affected the ability of the human subject tocorrectly classify plots as structured or noisy. The can requireinterpretation. Data points rendered in white with a black backgroundcan be more digestible than plots with a white background and coloreddata.

In an experiment of the human-machine co-learning system 300 initiallyrendered the data as a simple scatter plot showing every point of thedata set, as shown in inset (A) of FIG. 4C. The process of showing highresolution scatterplots, at a rate of ten per second, proved to create amovie-like effect, and the perceived “motion” of the data points betweenplots became a confusing distraction for the operator. This wasaddressed by rendering the data as a low resolution, two-dimensionalhistogram instead, as shown in inset (B) of FIG. 4C. The lowerresolution plots still accurately represented the potential structure inthe data and proved easier for the operator to process at 10 Hz becausethere was no perceived motion of the points. Based upon the initialexperiments with RSVP, it was shown that minimalistic, soft-edgedvisualizations tend to perform better than nuanced, high-detail plots.

The human-machine co-learning system 300 can be configured to be asgenerally applicable to dataset interaction as possible. For example,one application is the exploration of telescope image datasets;distinguishing images that contain interesting astronomical phenomenafrom blurry or noisy images is a straightforward application of RSVP.Another example uses the human-machine co-learning system 300 todiscover outlier events in network traffic datasets. While displaying avisual playback of network traffic, RSVP can capture and filter momentsof network activity that interest the user 114.

Data visualization is a component to the success of data explorationwith the human-machine co-learning system 300, and it is clear that notall datasets are intuitively visualized in a meaningful way. Thehuman-machine co-learning system 300 may use RSVP to discoverinteresting and unknown visualizations of datasets, particularly networktraffic data. While investigating a dataset, the data renderer coulditerate through and display a host of different visualizations of thesame data; a particular visualization technique could reveal someunknown structure that intrigues the user 114. From the EEG signals, thehuman-machine co-learning system 300 can capture this human interest andmark the particular visualization for further use. While at a glancethis trial-and-error discovery of effective visualizations may not seemvery efficient, the speed of interaction between human and computer viaRSVP allows the human-machine co-learning system 300 to process hundredsof different visualizations over the course of a minute.

The foregoing description is illustrative, and variations inconfiguration and implementation can occur to persons skilled in theart. For instance, the various illustrative logics, logical blocks,modules, and circuits described in connection with the embodimentsdisclosed herein can be implemented or performed with a general purposeprocessor, a digital signal processor (DSP), an application specificintegrated circuit (ASIC), a field programmable gate array (FPGA) orother programmable logic device, discrete gate or transistor logic,discrete hardware components, or any combination thereof designed toperform the functions described herein. A general-purpose processor canbe a microprocessor, but, in the alternative, the processor can be anyconventional processor, controller, microcontroller, or state machine. Aprocessor can also be implemented as a combination of computing devices,e.g., a combination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration.

In one or more exemplary embodiments, the functions described can beimplemented in hardware, software, firmware, or any combination thereof.For a software implementation, the techniques described herein can beimplemented with modules (e.g., procedures, functions, subprograms,programs, routines, subroutines, modules, software packages, classes,and so on) that perform the functions described herein. A module can becoupled to another module or a hardware circuit by passing and/orreceiving information, data, arguments, parameters, or memory contents.Information, arguments, parameters, data, or the like can be passed,forwarded, or transmitted using any suitable means including memorysharing, message passing, token passing, network transmission, and thelike. The software codes can be stored in memory units and executed byprocessors. The memory unit can be implemented within the processor orexternal to the processor, in which case it can be communicativelycoupled to the processor via various means as is known in the art.

For example, FIG. 5 illustrates an example of a hardware configurationfor the computer system 102 in the human-machine co-learning system 100.While FIG. 5 illustrates various components contained in the computerdevice 500, FIG. 5 illustrates one example of a computer device andadditional components can be added and existing components can beremoved.

The computer device 500 can be any type of computer device. Asillustrated in FIG. 5, the computer device 500 can include one or moreprocessors 502 of varying core configurations and clock frequencies. Thecomputer device 500 can also include one or more memory devices 504 thatserve as a main memory during the operation of the computer device 500.For example, during operation, a copy of the software that supports thedata analysis tool 110 can be stored in the one or more memory devices504. The computer device 500 can also include one or more peripheralinterfaces 506, such as keyboards, mice, touchpads, computer screens,touchscreens, etc., for enabling human interaction with and manipulationof the computer device 500.

The computer device 500 can also include one or more network interfaces508 for communicating via one or more networks, for example the network108, such as Ethernet adapters, wireless transceivers, or serial networkcomponents, for communicating over wired or wireless media usingprotocols. The computer device 500 can also include one or more storagedevice 510 of varying physical dimensions and storage capacities, suchas flash drives, hard drives, random access memory, etc., for storingdata, such as images, files, and program instructions for execution bythe one or more processors 502.

Additionally, the computer device 500 can include one or more softwareprograms 512 that enable the functionality of the data analysis tool 110described above. The one or more software programs 512 can includeinstructions that cause the one or more processors 502 to perform theprocesses described herein. Copies of the one or more software programs512 can be stored in the one or more memory devices 504 and/or on in theone or more storage devices 510. Likewise, the data utilized by one ormore software programs 512 can be stored in the one or more memorydevices 504 and/or on in the one or more storage devices 510.

The computer device 500 can include a variety of data stores and othermemory and storage media as discussed above. These can reside in avariety of locations, such as on a storage medium local to (and/orresident in) one or more of the computers or remote from any or all ofthe computers across the network. In some implementations, informationcan reside in a storage-area network (SAN) familiar to those skilled inthe art. Similarly, any necessary files for performing the functionsattributed to the computers, servers, or other network devices may bestored locally and/or remotely, as appropriate.

In implementations, the components of the computer device 500 asdescribed above need not be enclosed within a single enclosure or evenlocated in close proximity to one another. Those skilled in the art willappreciate that the above-described componentry are examples only, asthe computer device 500 can include any type of hardware componentry,including any necessary accompanying firmware or software, forperforming the disclosed implementations. The computer device 500 canalso be implemented in part or in whole by electronic circuit componentsor processors, such as application-specific integrated circuits (ASICs)or field-programmable gate arrays (FPGAs).

If implemented in software, the functions can be stored on ortransmitted over a computer-readable medium as one or more instructionsor code. Computer-readable media includes both tangible, non-transitorycomputer storage media and communication media including any medium thatfacilitates transfer of a computer program from one place to another. Astorage media can be any available tangible, non-transitory media thatcan be accessed by a computer. By way of example, and not limitation,such tangible, non-transitory computer-readable media can comprise arandom access memory (RAM), a read only memory (ROM), a flash memory, anelectrically erasable programmable read only memory (EEPROM), a compactdisc read only memory (CD-ROM) or other optical disk storage, magneticdisk storage or other magnetic storage devices, or any other medium thatcan be used to carry or store desired program code in the form ofinstructions or data structures and that can be accessed by a computer.Disk and disc, as used herein, includes CD, laser disc, optical disc,digital versatile disc (DVD), floppy disk and Blu-ray disc where disksusually reproduce data magnetically, while discs reproduce dataoptically with lasers. Also, any connection is properly termed acomputer-readable medium. For example, if the software is transmittedfrom a website, server, or other remote source using a coaxial cable,fiber optic cable, twisted pair, digital subscriber line (DSL), orwireless technologies such as infrared, radio, and microwave, then thecoaxial cable, fiber optic cable, twisted pair, DSL, or wirelesstechnologies such as infrared, radio, and microwave are included in thedefinition of medium. Combinations of the above should also be includedwithin the scope of computer-readable media.

As non-limiting examples unless specifically indicated, any repository,database, or data store described herein can comprise a local database,online database, desktop database, server-side database, relationaldatabase, hierarchical database, network database, object database,object-relational database, associative database, concept-orienteddatabase, entity-attribute-value database, multidimensional database,semi-structured database, star schema database, XML database, file,collection of files, spreadsheet, or other means of data storage locatedon a computer, client, server, or any other storage device known in theart or developed in the future. File systems for file or any repository,database, or data store can be any file system, including withoutlimitation disk or shared disk, flash, tape, database, transactional,and network file systems, using UNIX, Linux, Mac OS X, Windows FAT orNTFS, FreeBSD, or any other operating system.

While the teachings have been described with reference to examples ofthe implementations thereof, those skilled in the art will be able tomake various modifications to the described implementations withoutdeparting from the true spirit and scope. The terms and descriptionsused herein are set forth by way of illustration only and are not meantas limitations. In particular, although the processes have beendescribed by examples, the stages of the processes can be performed in adifferent order than illustrated or simultaneously. Furthermore, to theextent that the terms “including”, “includes”, “having”, “has”, “with”,or variants thereof are used in the detailed description, such terms areintended to be inclusive in a manner similar to the term “comprising.”As used herein, the terms “one or more of” and “at least one of” withrespect to a listing of items such as, for example, A and B, means Aalone, B alone, or A and B. Further, unless specified otherwise, theterm “set” should be interpreted as “one or more.” Also, the term“couple” or “couples” is intended to mean either an indirect or directconnection. Thus, if a first device couples to a second device, thatconnection can be through a direct connection, or through an indirectconnection via other devices, components, and connections.

What is claimed is:
 1. A method, comprising: generating data renderingsfor a data set, wherein the data renderings for the data set compriseone or more of visual renderings of portions of the data set and one ormore sonic renderings of portions of the data set; providing the datarenderings to a user via one or more output devices; capturingbiofeedback data from the user using one or more human interfacedevices, wherein the biofeedback data comprises biological responses tothe one or more data renderings; continuously generating and providingnew data renderings based on the biofeedback data, wherein the new datarenderings incorporate features in the data renderings identified fromthe biofeedback data; and determining one or more features of interestin the data set based on the biofeedback data and the new datarenderings.
 2. The method of claim 1, wherein the new data renderingsare iteratively generated based on the biofeedback data and wherein thenew data renderings are provided continuously to the user via the one ormore output devices.
 3. The method of claim 2, wherein continuouslygenerating and providing the new data renderings comprises: displaying,on a display device, one or more of visual renderings of the new datarenderings at a high display rate.
 4. The method of claim 2, wherein thenew data renderings are iteratively generated until the one or morefeatures of interest is identified in the data set.
 5. The method ofclaim 1, wherein the new data renderings comprises new visual renderingsof a reduced portion of the data set.
 6. The method of claim 1, themethod further comprising: training a machine learning algorithm usingthe biofeedback data and the new data renderings.
 7. The method of claim1, wherein the biofeedback data comprises brain wave data, eye movementtracking data, micro-facial expressions data, body language, vital signdata, muscle movement data, capillary dilation data, and skinconductivity data.
 8. A system, comprising: one or more output devices;one or more human interface devices; and a computer system coupled tothe one or more output devices and the one or more human interfacedevices, wherein the computer system comprises one or more memorydevices storing instructions and one or more processors coupled to theone or more memory devices and configured to execute the instructions toperform a method comprising: generating data renderings for a data set,wherein the data renderings for the data set comprise one or more ofvisual renderings of portions of the data set and one or more sonicrenderings of portions of the data set; providing the data renderings toa user via the one or more output devices; capturing biofeedback datafrom the user using the one or more human interface devices, wherein thebiofeedback data comprises biological responses to the one or more datarenderings; continuously generating and providing new data renderingsbased on the biofeedback data, wherein the new data renderingsincorporate features in the data renderings identified from thebiofeedback data; and determining one or more features of interest inthe data set based on the biofeedback data and the new data renderings.9. The system of claim 8, wherein the new data renderings areiteratively generated based on the biofeedback data and wherein the newdata renderings are provided continuously to the user via the one ormore output devices.
 10. The system of claim 9, wherein continuouslygenerating and providing the new data renderings comprises: displaying,on a display device, one or more of visual renderings of the new datarenderings at a high display rate.
 11. The system of claim 9, whereinthe new data renderings are iteratively generated until the one or morefeatures of interest is identified in the data set.
 12. The system ofclaim 8, wherein the new data renderings comprises new visual renderingsof a reduced portion of the data set.
 13. The system of claim 8, themethod further comprising: training a machine learning algorithm usingthe biofeedback data and the new data renderings.
 14. The system ofclaim 8, wherein the biofeedback data comprises brain wave data, eyemovement tracking data, micro-facial expressions data, body language,vital sign data, muscle movement data, capillary dilation data, and skinconductivity data.
 15. A non-transitory computer readable medium storinginstructions for causing one or more processors to perform a method, themethod comprising: generating data renderings for a data set, whereinthe data renderings for the data set comprise one or more of visualrenderings of portions of the data set and one or more sonic renderingsof portions of the data set; providing the data renderings to a user viaone or more output devices; capturing biofeedback data from the userusing one or more human interface devices, wherein the biofeedback datacomprises biological responses to the one or more data renderings;continuously generating and providing new data renderings based on thebiofeedback data, wherein the new data renderings incorporate featuresin the data renderings identified from the biofeedback data; anddetermining one or more features of interest in the data set based onthe biofeedback data and the new data renderings.
 16. The non-transitorycomputer readable medium of claim 15, wherein the new data renderingsare iteratively generated based on the biofeedback data and wherein thenew data renderings are provided continuously to the user via the one ormore output devices.
 17. The non-transitory computer readable medium ofclaim 16, wherein continuously generating and providing the new datarenderings comprises: displaying, on a display device, one or more ofvisual renderings of the new data renderings at a high display rate. 18.The non-transitory computer readable medium of claim 16, wherein the newdata renderings are iteratively generated until the one or more featuresof interest is identified in the data set.
 19. The non-transitorycomputer readable medium of claim 15, wherein the new data renderingscomprises new visual renderings of a reduced portion of the data set.20. The non-transitory computer readable medium of claim 15, the methodfurther comprising: training a machine learning algorithm using thebiofeedback data and the new data renderings.